{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module 1 - Text Generation with Amazon Bedrock\n",
    "\n",
    "----\n",
    "\n",
    "This notebook provides sample code with step by step instructions for using Amazon Bedrock's **Converse API**.\n",
    "\n",
    "----\n",
    "\n",
    "### Contents\n",
    "\n",
    "1. [Setup](#ch1)\n",
    "2. [Text Summarization with Foundational Models](#ch2)\n",
    "   1. [Text Summarization using the Invoke Model API](#ch21)\n",
    "   2. [Text Summarization using the Converse API](#ch22)\n",
    "   3. [Overview of the Converse API](#ch23)\n",
    "   4. [Easily switch between different models](#ch24)\n",
    "   5. [Cross-Regional Inference in Amazon Bedrock](#ch25)\n",
    "   6. [Multi-turn conversations](#ch26)\n",
    "   7. [Streaming Responses with the ConverseStream API](#ch27)\n",
    "3. [Code Generation wiht Foundational Models](#ch3)\n",
    "4. [Function Calling with the Converse API](#ch4)\n",
    "   1. [4.1 Function Calling Flow](#ch41)\n",
    "   2. [Example: Implementation of Weather Function Calling with Bedrock's Converse API](#ch42)\n",
    "5. [Conclusion](#ch5)\n",
    "\n",
    "----\n",
    "\n",
    "### Introduction\n",
    "\n",
    "Welcome to this introduction to building conversational AI with Amazon Bedrock's Converse API! The primary goal of this chapter is to provide a comprehensive introduction to Amazon Bedrock APIs for generating text. While we'll explore various use cases like summarization and code generation, our focus is on understanding the API patterns.\n",
    "\n",
    "In this notebook, you will:\n",
    "\n",
    "1. Learn the basics of the Amazon Bedrock **Invoke API**\n",
    "2. Explore the more powerful **Converse API** and it's features like multi-turn conversation, streaming, or function calling\n",
    "3. Apply these APIs across various foundation models\n",
    "4. Compare results across different state-of-the-art models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-requisites\n",
    "This notebook requires permissions to:\n",
    "- Access **Amazon Bedrock**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Setup <a id=\"ch1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import boto3\n",
    "import botocore\n",
    "from IPython.display import display, Markdown\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Initial setup for clients, global variables and helper functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize Bedrock client\n",
    "session = boto3.session.Session()\n",
    "region = session.region_name\n",
    "bedrock = boto3.client(service_name='bedrock-runtime', region_name=region)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define model IDs that will be used in this module\n",
    "MODELS = {\n",
    "    \"Claude 3.7 Sonnet\": \"us.anthropic.claude-3-7-sonnet-20250219-v1:0\",\n",
    "    \"Claude 3.5 Sonnet\": \"us.anthropic.claude-3-5-sonnet-20240620-v1:0\",\n",
    "    \"Claude 3.5 Haiku\": \"us.anthropic.claude-3-5-haiku-20241022-v1:0\",\n",
    "    \"Amazon Nova Pro\": \"us.amazon.nova-pro-v1:0\",\n",
    "    \"Amazon Nova Micro\": \"us.amazon.nova-micro-v1:0\",\n",
    "    \"Meta Llama 3.1 70B Instruct\": \"us.meta.llama3-1-70b-instruct-v1:0\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Utility function to display model responses in a more readable format\n",
    "def display_response(response, model_name=None):\n",
    "    if model_name:\n",
    "        display(Markdown(f\"### Response from {model_name}\"))\n",
    "    display(Markdown(response))\n",
    "    print(\"\\n\" + \"-\"*80 + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Text Summarization with Foundation Models <a id=\"ch2\"></a>\n",
    "\n",
    "Let's start by exploring how to leverage Amazon Bedrock APIs for text summarization. We'll first use the basic Invoke API, then introduce the more powerful Converse API.\n",
    "\n",
    "As an example, let's take a paragraph about Amazon Bedrock from an [AWS blog post](https://aws.amazon.com/jp/blogs/machine-learning/announcing-new-tools-for-building-with-generative-ai-on-aws/).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_to_summarize = \"\"\"\n",
    "AWS took all of that feedback from customers, and today we are excited to announce Amazon Bedrock, \\\n",
    "a new service that makes FMs from AI21 Labs, Anthropic, Stability AI, and Amazon accessible via an API. \\\n",
    "Bedrock is the easiest way for customers to build and scale generative AI-based applications using FMs, \\\n",
    "democratizing access for all builders. Bedrock will offer the ability to access a range of powerful FMs \\\n",
    "for text and images—including Amazons Titan FMs, which consist of two new LLMs we're also announcing \\\n",
    "today—through a scalable, reliable, and secure AWS managed service. With Bedrock's serverless experience, \\\n",
    "customers can easily find the right model for what they're trying to get done, get started quickly, privately \\\n",
    "customize FMs with their own data, and easily integrate and deploy them into their applications using the AWS \\\n",
    "tools and capabilities they are familiar with, without having to manage any infrastructure (including integrations \\\n",
    "with Amazon SageMaker ML features like Experiments to test different models and Pipelines to manage their FMs at scale).\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Text Summarization using the Invoke Model API <a id=\"ch21\"></a>\n",
    "\n",
    "Amazon Bedrock's **Invoke Model API** serves as the most basic method for sending requests to foundation models. Since each model family has its own distinct request and response format, you'll need to craft specific JSON payloads tailored to each model.\n",
    "\n",
    "For this example, we will call Claude 3.7 Sonnet via Invoke Model API (using the `invoke_model` function of the Bedrock Runtime Client) to generate a summary of our text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create prompt for summarization\n",
    "prompt = f\"\"\"Please provide a summary of the following text. Do not add any information that is not mentioned in the text below.\n",
    "<text>\n",
    "{text_to_summarize}\n",
    "</text>\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create request body for Claude 3.7 Sonnet\n",
    "claude_body = json.dumps({\n",
    "    \"anthropic_version\": \"bedrock-2023-05-31\",\n",
    "    \"max_tokens\": 1000,\n",
    "    \"temperature\": 0.5,\n",
    "    \"top_p\": 0.9,\n",
    "    \"messages\": [\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [{\"type\": \"text\", \"text\": prompt}]\n",
    "        }\n",
    "    ],\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Send request to Claude 3.7 Sonnet\n",
    "try:\n",
    "    response = bedrock.invoke_model(\n",
    "        modelId=MODELS[\"Claude 3.7 Sonnet\"],\n",
    "        body=claude_body,\n",
    "        accept=\"application/json\",\n",
    "        contentType=\"application/json\"\n",
    "    )\n",
    "    response_body = json.loads(response.get('body').read())\n",
    "    \n",
    "    # Extract and display the response text\n",
    "    claude_summary = response_body[\"content\"][0][\"text\"]\n",
    "    display_response(claude_summary, \"Claude 3.7 Sonnet (Invoke Model API)\")\n",
    "    \n",
    "except botocore.exceptions.ClientError as error:\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "        print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "            \\nTo troubleshoot this issue please refer to the following resources.\\\n",
    "            \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "            \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "    else:\n",
    "        raise error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Text Summarization using the Converse API (Recommended Approach) <a id=\"ch22\"></a>\n",
    "\n",
    "While the **Invoke Model API** allows direct access to foundation models, it has several limitations:\n",
    "1. it uses different request/response formats for each model family;\n",
    "2. there is no built-in support for multi-turn conversations;\n",
    "3. it requires custom handling for different model capabilities.\n",
    "\n",
    "The **Converse API** addresses these limitations by providing a unified interface. Let's explore it on our text summarization task:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a converse request with our summarization task\n",
    "converse_request = {\n",
    "    \"messages\": [\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [\n",
    "                {\n",
    "                    \"text\": f\"Please provide a concise summary of the following text in 2-3 sentences. Text to summarize: {text_to_summarize}\"\n",
    "                }\n",
    "            ]\n",
    "        }\n",
    "    ],\n",
    "    \"inferenceConfig\": {\n",
    "        \"temperature\": 0.4,\n",
    "        \"topP\": 0.9,\n",
    "        \"maxTokens\": 500\n",
    "    }\n",
    "}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Call Claude 3.7 Sonnet with Converse API\n",
    "try:\n",
    "    response = bedrock.converse(\n",
    "        modelId=MODELS[\"Claude 3.7 Sonnet\"],\n",
    "        messages=converse_request[\"messages\"],\n",
    "        inferenceConfig=converse_request[\"inferenceConfig\"]\n",
    "    )\n",
    "    \n",
    "    # Extract the model's response\n",
    "    claude_converse_response = response[\"output\"][\"message\"][\"content\"][0][\"text\"]\n",
    "    display_response(claude_converse_response, \"Claude 3.7 Sonnet (Converse API)\")\n",
    "except botocore.exceptions.ClientError as error:\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "        print(f\"\\x1b[41m{error.response['Error']['Code']}: {error.response['Error']['Message']}\\x1b[0m\")\n",
    "        print(\"Please ensure you have the necessary permissions for Amazon Bedrock.\")\n",
    "    else:\n",
    "        raise error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Overview of the Converse API <a id=\"ch23\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, that we have used the **Converse API**, let's take some time to take a closer look. To use the Converse API, you use the <a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html\" target=\"_blank\">Converse</a> or <a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html\" target=\"_blank\">ConverseStream</a> (for streaming responses) operations to send messages to a model. While, it is possible to use the existing base inference operations (<a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html\" target=\"_blank\">InvokeModel</a> or <a href=\"https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html\" target=\"_blank\">InvokeModelWithResponseStream</a>) for conversation applications as well, we recommend using the Converse API as it provides consistent API, that works with <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/converse-api.html\" target=\"_blank\">all Amazon Bedrock models that support messages</a>. This means you can write code once and use it with different models. Should a model have unique inference parameters, the Converse API also allows you to pass those unique parameters in a model specific structure. You can use the Amazon Bedrock Converse API to create conversational applications that send and receive messages to and from an Amazon Bedrock model. For example, you can create a chat bot that maintains a conversation over many turns and uses a persona or tone customization that is unique to your needs, such as a helpful technical support assistant. The Converse API also supports other Bedrock capabilites, like <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html\" target=\"_blank\">tool use</a> and <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-converse-api.html\" target=\"_blank\">guardrails</a>.\n",
    "\n",
    "Let's break down its key components (you can also review the <a href=\"https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-runtime/client/converse.html\" target=\"_blank\">documentation</a> for a full list of parameters):\n",
    "\n",
    "```json\n",
    "{\n",
    "  \"modelId\": \"us.anthropic.claude-3-7-sonnet-20250219-v1:0\", // Required: Model identifier\n",
    "  \n",
    "  \"messages\": [ // Required: Conversation history\n",
    "    {\n",
    "      \"role\": \"user\", // Who sent the message\n",
    "      \"content\": [\n",
    "        {\n",
    "          \"text\": \"Your prompt or message here\" // Message content\n",
    "        }\n",
    "      ]\n",
    "    }\n",
    "  ],\n",
    "  \n",
    "  \"system\": [ // Optional: System instructions\n",
    "    {\n",
    "      \"text\": \"You are a helpful AI assistant.\"\n",
    "    }\n",
    "  ],\n",
    "  \n",
    "  \"inferenceConfig\": { // Optional: Inference parameters\n",
    "    \"temperature\": 0.7, // Randomness (0.0-1.0)\n",
    "    \"topP\": 0.9, // Diversity control (0.0-1.0)\n",
    "    \"maxTokens\": 2000, // Maximum response length\n",
    "    \"stopSequences\": [] // Stop generation triggers\n",
    "  },\n",
    "  \n",
    "  \"toolConfig\": { // Optional: Function calling setup\n",
    "    \"tools\": [],\n",
    "    \"toolChoice\": {\n",
    "      \"auto\": {} // Let model decide when to use tools\n",
    "    }\n",
    "  }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Easily switch between models <a id=\"ch24\"></a>\n",
    "One of the biggest advantages of the Converse API is the ability to easily switch between models using the exact same request format. Let's compare summaries across different foundation models by looping over the model dictionary we defined above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# call different models with the same converse request\n",
    "results = {}    \n",
    "for model_name, model_id in MODELS.items(): # looping over all models defined above\n",
    "        try:\n",
    "            start_time = time.time()\n",
    "            response = bedrock.converse(\n",
    "                modelId=model_id,\n",
    "                messages=converse_request[\"messages\"],\n",
    "                inferenceConfig=converse_request[\"inferenceConfig\"] if \"inferenceConfig\" in converse_request else None\n",
    "            )\n",
    "            end_time = time.time()\n",
    "            \n",
    "            # Extract the model's response using the correct structure\n",
    "            model_response = response[\"output\"][\"message\"][\"content\"][0][\"text\"]\n",
    "            response_time = round(end_time - start_time, 2)\n",
    "            \n",
    "            results[model_name] = {\n",
    "                \"response\": model_response,\n",
    "                \"time\": response_time\n",
    "            }\n",
    "            \n",
    "            print(f\"✅ Successfully called {model_name} (took {response_time} seconds)\")\n",
    "            \n",
    "        except Exception as e:\n",
    "            print(f\"❌ Error calling {model_name}: {str(e)}\")\n",
    "            results[model_name] = {\n",
    "                \"response\": f\"Error: {str(e)}\",\n",
    "                \"time\": None\n",
    "            }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display results in a formatted way\n",
    "for model_name, result in results.items():\n",
    "    if \"Error\" not in result[\"response\"]:\n",
    "        display(Markdown(f\"### {model_name} (took {result['time']} seconds)\"))\n",
    "        display(Markdown(result[\"response\"]))\n",
    "        print(\"-\" * 80)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 Cross-Regional Inference in Amazon Bedrock <a id=\"ch25\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Amazon Bedrock offers Cross-Regional Inference which automatically selects the optimal AWS Region within your geography to process your inference requests. Cross-Regional Inference offers higher throughput limits (up to 2x allocated quotas) and seamlessly manages traffic bursts by dynamically routing requests across multiple AWS regions, enhancing application resilience during peak demand periods without additional routing or data transfer costs. Customers can control where their inference data flows by selecting from a pre-defined set of regions, helping them comply with applicable data residency requirements and sovereignty laws. Moreover, this capability prioritizes the connected Bedrock API source region when possible, helping to minimize latency and improve responsiveness. As a result, customers can enhance their applications' reliability, performance, and efficiency. Please review the list of <a href=\"https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html\" target=\"_blank\">supported regions and models for inference profiles</a>. \n",
    "\n",
    "To use Cross-Regional Inference, you simply need to specify a cross-region inference profile as the `modelId` when making a request. Cross-region inference profiles are identified by including a region prefix (e.g., `us.` or `eu.`) before the model name.\n",
    "\n",
    "For example:\n",
    "```json\n",
    "{\n",
    "    \"Amazon Nova Pro\": \"amazon.nova-pro-v1:0\",  # Regular model ID\n",
    "    \"Amazon Nova Pro (CRIS)\": \"us.amazon.nova-pro-v1:0\"  # Cross-regional model ID\n",
    "}\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how easy it is to use Cross Region Inference by invoking the Claude 3.5 Sonnet model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Regular model invocation (standard region)\n",
    "standard_response = bedrock.converse(\n",
    "    modelId=\"anthropic.claude-3-5-sonnet-20240620-v1:0\",  # Standard model ID\n",
    "    messages=converse_request[\"messages\"]\n",
    ")\n",
    "\n",
    "# Cross-region inference (note the \"us.\" prefix)\n",
    "cris_response = bedrock.converse(\n",
    "    modelId=\"us.anthropic.claude-3-5-sonnet-20240620-v1:0\",  # Cross-region model ID with regional prefix\n",
    "    messages=converse_request[\"messages\"]\n",
    ")\n",
    "\n",
    "# Print responses\n",
    "print(\"Standard response:\", standard_response[\"output\"][\"message\"][\"content\"][0][\"text\"])\n",
    "print(\"Cross-region response:\", cris_response[\"output\"][\"message\"][\"content\"][0][\"text\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.6 Multi-turn Conversations <a id=\"ch26\"></a>\n",
    "The Converse API makes multi-turn conversations simple. Let's see it in action:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example of a multi-turn conversation with Converse API\n",
    "multi_turn_messages = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [{\"text\": f\"Please summarize this text: {text_to_summarize}\"}]\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"assistant\",\n",
    "        \"content\": [{\"text\": results[\"Claude 3.7 Sonnet\"][\"response\"]}]\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [{\"text\": \"Can you make this summary even shorter, just 1 sentence?\"}]\n",
    "    }\n",
    "]\n",
    "\n",
    "try:\n",
    "    response = bedrock.converse(\n",
    "        modelId=MODELS[\"Claude 3.7 Sonnet\"],\n",
    "        messages=multi_turn_messages,\n",
    "        inferenceConfig={\"temperature\": 0.2, \"maxTokens\": 500}\n",
    "    )\n",
    "    \n",
    "    # Extract the model's response using the correct structure\n",
    "    follow_up_response = response[\"output\"][\"message\"][\"content\"][0][\"text\"]\n",
    "    display_response(follow_up_response, \"Claude 3.7 Sonnet (Multi-turn conversation)\")\n",
    "    \n",
    "except Exception as e:\n",
    "    print(f\"Error: {str(e)}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.7 Streaming Responses with ConverseStream API <a id=\"ch27\"></a>\n",
    "\n",
    "For longer generations, you might want to receive the content as it's being generated. The ConverseStream API supports streaming, which allows you to process the response incrementally:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example of streaming with Converse API\n",
    "def stream_converse(model_id, messages, inference_config=None):\n",
    "    if inference_config is None:\n",
    "        inference_config = {}\n",
    "    \n",
    "    print(\"Streaming response (chunks will appear as they are received):\\n\")\n",
    "    print(\"-\" * 80)\n",
    "    \n",
    "    full_response = \"\"\n",
    "    \n",
    "    try:\n",
    "        response = bedrock.converse_stream(\n",
    "            modelId=model_id,\n",
    "            messages=messages,\n",
    "            inferenceConfig=inference_config\n",
    "        )\n",
    "        response_stream = response.get('stream')\n",
    "        if response_stream:\n",
    "            for event in response_stream:\n",
    "\n",
    "                if 'messageStart' in event:\n",
    "                    print(f\"\\nRole: {event['messageStart']['role']}\")\n",
    "\n",
    "                if 'contentBlockDelta' in event:\n",
    "                    print(event['contentBlockDelta']['delta']['text'], end=\"\")\n",
    "\n",
    "                if 'messageStop' in event:\n",
    "                    print(f\"\\nStop reason: {event['messageStop']['stopReason']}\")\n",
    "\n",
    "                if 'metadata' in event:\n",
    "                    metadata = event['metadata']\n",
    "                    if 'usage' in metadata:\n",
    "                        print(\"\\nToken usage\")\n",
    "                        print(f\"Input tokens: {metadata['usage']['inputTokens']}\")\n",
    "                        print(\n",
    "                            f\":Output tokens: {metadata['usage']['outputTokens']}\")\n",
    "                        print(f\":Total tokens: {metadata['usage']['totalTokens']}\")\n",
    "                    if 'metrics' in event['metadata']:\n",
    "                        print(\n",
    "                            f\"Latency: {metadata['metrics']['latencyMs']} milliseconds\")\n",
    "\n",
    "                \n",
    "            print(\"\\n\" + \"-\" * 80)\n",
    "        return full_response\n",
    "    \n",
    "    except Exception as e:\n",
    "        print(f\"Error in streaming: {str(e)}\")\n",
    "        return None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's try streaming a longer summary\n",
    "streaming_request = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [\n",
    "            {\n",
    "                \"text\": f\"\"\"Please provide a detailed summary of the following text, explaining its key points and implications:\n",
    "                \n",
    "                {text_to_summarize}\n",
    "                \n",
    "                Make your summary comprehensive but clear.\n",
    "                \"\"\"\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only run this when you're ready to see streaming output\n",
    "streamed_response = stream_converse(\n",
    "    MODELS[\"Claude 3.7 Sonnet\"], \n",
    "    streaming_request, \n",
    "    inference_config={\"temperature\": 0.4, \"maxTokens\": 1000}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Code Generation with FMs <a id=\"ch3\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Foundation models like Claude excel not only at generating prose but also at creating programming language code, making them valuable tools for engineering work automation. In this chapter, we'll use Claude and the Bedrock Converse API to generate code for a weather function that we'll in the next chapter integrate as a tool for our models. We'll craft a specific prompt for Claude, configure and execute a Bedrock Converse API request, and save the generated weather function with mock data for use in the next chapter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prompt the LLM to write us a get_weather function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "code_generation_prompt = \"\"\"\n",
    "Create a Python function called get_weather that accepts a location as parameter. \\\n",
    "The function should return a dictionary containing weather data (condition, temperature, and humidity) for predefined cities.\\\n",
    "Use a mock data structure instead of actual API calls. Include New York, San Francisco, Miami, and Seattle as default cities.\\\n",
    "The return statement should look like the following: return weather_data.get(location, {\"condition\": \"Unknown\", \"temperature\": 0, \"humidity\": 0}).\n",
    "Only return the function and no preamble or examples.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a converse request with our code generation task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "converse_request = {\n",
    "    \"messages\": [\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [\n",
    "                {\n",
    "                    \"text\": f\"{code_generation_prompt}\"\n",
    "                }\n",
    "            ]\n",
    "        }\n",
    "    ],\n",
    "    \"inferenceConfig\": {\n",
    "        \"temperature\": 0.0,\n",
    "        \"topP\": 0.9,\n",
    "        \"maxTokens\": 500\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Call Claude 3.7 Sonnet with Converse API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    response = bedrock.converse(\n",
    "        modelId=MODELS[\"Claude 3.7 Sonnet\"],\n",
    "        messages=converse_request[\"messages\"],\n",
    "        inferenceConfig=converse_request[\"inferenceConfig\"]\n",
    "    )\n",
    "    \n",
    "    # Extract the model's response\n",
    "    claude_converse_response = response[\"output\"][\"message\"][\"content\"][0][\"text\"]\n",
    "    display_response(claude_converse_response, \"Claude 3.7 Sonnet (Converse API)\")\n",
    "except botocore.exceptions.ClientError as error:\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "        print(f\"\\x1b[41m{error.response['Error']['Code']}: {error.response['Error']['Message']}\\x1b[0m\")\n",
    "        print(\"Please ensure you have the necessary permissions for Amazon Bedrock.\")\n",
    "    else:\n",
    "        raise error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now lets copy the created function to the below cell define it in our notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copy-Paste the created get_weather in this cell:\n",
    "# def get_weather(location): ...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Function Calling with Amazon Bedrock <a id=\"ch4\"></a>\n",
    "\n",
    "Modern LLMs like Claude go beyond generating free-form text — they can also reason about when external tools or functions should be used to better answer user questions. This capability, known as _function calling_ (or _tool use_), enables the model to decide _which_ function to call, _when_ to call it, and _what_ parameters to provide — but importantly, the model does not execute the function itself.\n",
    "\n",
    "Instead, the model returns a well-structured response (typically in JSON format) that describes the intended function call. It’s then up to your application to detect this output, execute the requested function (such as calling an API or querying a database), and pass the result back to the model — allowing it to generate a final, user-friendly response that incorporates real-world data.\n",
    "\n",
    "Function calling is especially useful when building LLM-powered applications that need access to dynamic, external information — for example, retrieving real-time weather data, which is exactly what we’ll demonstrate in the this section.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Function Calling Flow <a id=\"ch41\"></a>\n",
    "Amazon Bedrock natively supports function calling through its Converse API, which provides a consistent way to give LLMs access to tools or functions that live outside the model.\n",
    "\n",
    "The typical function calling flow looks like this:\n",
    "\n",
    "- **Step 1 — Function Recognition**. When the model (e.g., Claude 3.7) identifies that a tool should be used to answer a user query (e.g., _\"What's the weather in Seattle?\"_), it returns a structured response indicating:\n",
    "\n",
    "  - The function name to call (e.g., `get_weather`)\n",
    "\n",
    "  - The required input parameters (e.g., `location=\"Seattle\"`)\n",
    "\n",
    "- **Step 2 — Function Execution** (by your application). The application is responsible for:\n",
    "\n",
    "  - Executing the requested function\n",
    "\n",
    "  - Capturing the output (e.g., current weather data)\n",
    "\n",
    "  - Passing the result back to the model\n",
    "\n",
    "- **Step 3 — Final Response Generation**. The LLM uses the function's output to generate a natural language response for the user."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Example: Implementation of Weather Function Calling with Bedrock's Converse API <a id=\"ch42\"></a>\n",
    "In this example, we’ll integrate a simple `get_weather` function with Claude 3.7 using the Converse API. The tool specification we provide to Claude defines:\n",
    "\n",
    "- The function name: `get_weather`\n",
    "\n",
    "- The function's purpose: _\"Retrieve weather for a given location\"_\n",
    "\n",
    "- The required input parameters: `location` (string)\n",
    "\n",
    "#### Example Workflow\n",
    "\n",
    "```mermaid\n",
    "User: \"What's the weather in Seattle?\"\n",
    "↓\n",
    "LLM: Returns function call request for `get_weather(\"Seattle\")`\n",
    "↓\n",
    "Application: Executes weather lookup, and feeds the result back to the LLM\n",
    "↓\n",
    "LLM: Generates response using actual weather data\n",
    "```\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Important:</b> Make sure you created the get_weather() function in one of the cells above (by copy-pasting the code that Claude has generated). In case something went wrong you can uncomment the back up implementation of the weather lookup function below:\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# def get_weather(location):\n",
    "#     \"\"\"Mock function that would normally call a weather API\"\"\"\n",
    "#     print(f\"Looking up weather for {location}...\")\n",
    "    \n",
    "#     # In a real application, this would call a weather API\n",
    "#     weather_data = {\n",
    "#         \"New York\": {\"condition\": \"Partly Cloudy\", \"temperature\": 72, \"humidity\": 65},\n",
    "#         \"San Francisco\": {\"condition\": \"Foggy\", \"temperature\": 58, \"humidity\": 80},\n",
    "#         \"Miami\": {\"condition\": \"Sunny\", \"temperature\": 85, \"humidity\": 75},\n",
    "#         \"Seattle\": {\"condition\": \"Rainy\", \"temperature\": 52, \"humidity\": 90}\n",
    "#     }\n",
    "    \n",
    "#     return weather_data.get(location, {\"condition\": \"Unknown\", \"temperature\": 0, \"humidity\": 0})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, lets define our tool specification.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "weather_tool = {\n",
    "    \"tools\": [\n",
    "        {\n",
    "            \"toolSpec\": {\n",
    "                \"name\": \"get_weather\",\n",
    "                \"description\": \"Get current weather for a specific location\",\n",
    "                \"inputSchema\": {\n",
    "                    \"json\": {\n",
    "                        \"type\": \"object\",\n",
    "                        \"properties\": {\n",
    "                            \"location\": {\n",
    "                                \"type\": \"string\",\n",
    "                                \"description\": \"The city name to get weather for\"\n",
    "                            }\n",
    "                        },\n",
    "                        \"required\": [\"location\"]\n",
    "                    }\n",
    "                }\n",
    "            }\n",
    "        }\n",
    "    ],\n",
    "    \"toolChoice\": {\n",
    "        \"auto\": {}  # Let the model decide when to use the tool\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ".. and create a simple function calling request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function_request = {\n",
    "    \"messages\": [\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [\n",
    "                {\n",
    "                    \"text\": \"What's the weather like in San Francisco right now? And what should I wear?\"\n",
    "                }\n",
    "            ]\n",
    "        }\n",
    "    ],\n",
    "    \"inferenceConfig\": {\n",
    "        \"temperature\": 0.0,  # Use 0 temperature for deterministic function calling\n",
    "        \"maxTokens\": 500\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the **tool specification** and the **function_request** we can now invoke the model and take a look it it's response. Observe how it correctly fills out the \"toolUse\" parameter in its response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "response = bedrock.converse(\n",
    "    modelId=MODELS[\"Claude 3.7 Sonnet\"],\n",
    "    messages=function_request[\"messages\"],\n",
    "    inferenceConfig=function_request[\"inferenceConfig\"],\n",
    "    toolConfig=weather_tool\n",
    ")\n",
    "print(json.dumps(response, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now build a function to handle the complete function calling flow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def handle_function_calling(model_id, request, tool_config):\n",
    "    try:\n",
    "        # Step 1: Send initial request\n",
    "        response = bedrock.converse(\n",
    "            modelId=model_id,\n",
    "            messages=request[\"messages\"],\n",
    "            inferenceConfig=request[\"inferenceConfig\"],\n",
    "            toolConfig=tool_config\n",
    "        )\n",
    "        \n",
    "        # Check if the model wants to use a tool (check the correct response structure)\n",
    "        content_blocks = response[\"output\"][\"message\"][\"content\"]\n",
    "        has_tool_use = any(\"toolUse\" in block for block in content_blocks)\n",
    "        \n",
    "        if has_tool_use:\n",
    "            # Find the toolUse block\n",
    "            tool_use_block = next(block for block in content_blocks if \"toolUse\" in block)\n",
    "            tool_use = tool_use_block[\"toolUse\"]\n",
    "            tool_name = tool_use[\"name\"]\n",
    "            tool_input = tool_use[\"input\"]\n",
    "            tool_use_id = tool_use[\"toolUseId\"]\n",
    "            \n",
    "            # Step 2: Execute the tool\n",
    "            if tool_name == \"get_weather\":\n",
    "                tool_result = get_weather(tool_input[\"location\"])\n",
    "            else:\n",
    "                tool_result = {\"error\": f\"Unknown tool: {tool_name}\"}\n",
    "            \n",
    "            # Step 3: Send the tool result back to the model\n",
    "            updated_messages = request[\"messages\"] + [\n",
    "                {\n",
    "                    \"role\": \"assistant\",\n",
    "                    \"content\": [\n",
    "                        {\n",
    "                            \"toolUse\": {\n",
    "                                \"toolUseId\": tool_use_id,\n",
    "                                \"name\": tool_name,\n",
    "                                \"input\": tool_input\n",
    "                            }\n",
    "                        }\n",
    "                    ]\n",
    "                },\n",
    "                {\n",
    "                    \"role\": \"user\",\n",
    "                    \"content\": [\n",
    "                        {\n",
    "                            \"toolResult\": {\n",
    "                                \"toolUseId\": tool_use_id,\n",
    "                                \"content\": [\n",
    "                                    {\n",
    "                                        \"json\": tool_result\n",
    "                                    }\n",
    "                                ],\n",
    "                                \"status\": \"success\"\n",
    "                            }\n",
    "                        }\n",
    "                    ]\n",
    "                }\n",
    "            ]\n",
    "            \n",
    "            # Step 4: Get final response\n",
    "            final_response = bedrock.converse(\n",
    "                modelId=model_id,\n",
    "                messages=updated_messages,\n",
    "                inferenceConfig=request[\"inferenceConfig\"],\n",
    "                toolConfig=tool_config  \n",
    "            )\n",
    "            \n",
    "            # Extract text from the correct response structure\n",
    "            final_text = \"\"\n",
    "            for block in final_response[\"output\"][\"message\"][\"content\"]:\n",
    "                if \"text\" in block:\n",
    "                    final_text = block[\"text\"]\n",
    "                    break\n",
    "            \n",
    "            return {\n",
    "                \"tool_call\": {\"name\": tool_name, \"input\": tool_input},\n",
    "                \"tool_result\": tool_result,\n",
    "                \"final_response\": final_text\n",
    "            }\n",
    "        else:\n",
    "            # Model didn't use a tool, just return the text response\n",
    "            text_response = \"\"\n",
    "            for block in content_blocks:\n",
    "                if \"text\" in block:\n",
    "                    text_response = block[\"text\"]\n",
    "                    break\n",
    "                    \n",
    "            return {\n",
    "                \"final_response\": text_response\n",
    "            }\n",
    "    \n",
    "    except Exception as e:\n",
    "        print(f\"Error in function calling: {str(e)}\")\n",
    "        return {\"error\": str(e)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the function calling flow with Claude 3.7 Sonnet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "function_result = handle_function_calling(\n",
    "    MODELS[\"Claude 3.7 Sonnet\"], \n",
    "    function_request,\n",
    "    weather_tool\n",
    ")\n",
    "\n",
    "# Display the results\n",
    "if \"error\" not in function_result:\n",
    "    if \"tool_call\" in function_result:\n",
    "        print(f\"Tool Call: {function_result['tool_call']['name']}({function_result['tool_call']['input']})\")\n",
    "        print(f\"Tool Result: {function_result['tool_result']}\")\n",
    "    \n",
    "    display_response(function_result[\"final_response\"], \"Claude 3.7 Sonnet (Function Calling)\")\n",
    "else:\n",
    "    print(f\"Error: {function_result['error']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Conclusion <a id=\"ch5\"></a>\n",
    "\n",
    "In this notebook, we've explored various capabilities of Amazon Bedrock APIs for the following use cases:\n",
    "\n",
    "1. **Text Summarization**: Using both Invoke and Converse APIs to create concise summaries\n",
    "2. **Model Flexibility**: Demonstrating easy switching between different foundation models\n",
    "3. **Multi-turn Conversations**: Building contextual dialogues using the Converse API\n",
    "4. **Streaming Responses**: Implementing real-time content delivery with ConverseStream API\n",
    "5. **Code Generation**: Using LLMs to generate functional code\n",
    "6. **Function Calling**: Integrating external tools to retrieve weather information for specific locations\n",
    "\n",
    "You are now prepared to move on to chapter 2, Knowledge Bases and RAG. Have fun!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
